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ABSTRACT 

This report presents the findings and implications of 
some research on the relationships of some educational inputs and 
reading performance of disadvantaged California Title I pupils which 
utilized a highly simplified model of the compensatory education 
process. The data for the study were gathered by questionnaire 
administered in personal interviews to a six percent sample of 
California Title I projects, enrolling ten percent of California 
Title I students. All projects in the study used the same performance 
measure, the Stanford Reading Test. Information was gathered and used 
for pupils in four elementary grades. The instructional variable 
which is roost consistently and dependably related to reading gains 
was that for the trained reading specialist. Specialist instruction 
was the only school input related to reading performance in the third 
grade considered separately, with the strength of the relationship 
even stronger in that grade than in the four pooled grades, while the 
variables for instruction by professionals were less statistically 
significant, they showed signs of having some importance. Of the 
variables constructed to measure planning and coordination, only that 
for hours of planning itself was related to performance. 

(Author/JM) 
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Page 4, 12th line from the top 

"and typically could not devote" should read: 
"and typically could only devote." 

Page 8, 

Table 1, first figure, first column should read: 

.076 months and not .76 months 



O 

ERIC 



a.'.l.TBmllU 



i Bid j 



SANTA MONICA CA 90406 







<r — * 

vD 

CO 

vO 

o 

o 

UJ 



SOME ESTIMATES FOR THE COST EFFECTIVENESS OF EDUCATIONAL INPUTS 
FOR READING PERFORMANCE OF DISADVANTAGED CHILDREN 
IN CALIFORNIA TITLE I PROJECTS 

A 

Herbert J. Kiesling" 

Consultant to The Rand Corporation, Santa Monica, California 
INTRODUCTION 

In a recently published Rand research report, ^ the author pre- 
sented relationships of some educational inputs and reading performance 
of disadvantaged Cal ifornia Title I pupils which utilized a highly simpli 
fied model of the compensatory education process. This paper is meant 
to give some extensions of those findings including some cost imputations 
It is meant to be helpful to practicing educational managers of compensa- 
tory education programs as well as to researchers. The numbers given are 
defended as being no more than highly suggestive. 

In order to prevent the reader from being required to read the origi 
nal report, the model and data used are summarized briefly in the first 
section. The basic hypotheses tested and findings are also given before 
some additional findings and cost relationships are presented. 

SUMMARY OF DATA, VARIABLES. AND WORKING HYPOTHESES 

The data for the study were gathered by questionnaire administered 

in personal interviews to a six-percent sample of California Title I 

[2] 

projects, enrolling ten percent of California Title I students. All 
projects in the study used the same performance measure, the Stanford 
Reading Test. Information was gathered and used for pupils in four ele- 
mentary grades. 

The following four hypotheses make up the basic framework or "model" 
about which the study was designed. 



Any views expressed in this paper are those of the author. They 
should not be interpreted as reflecting the views of The Rand Corporation 
or the official opinion or policy of any of its governmental or private 
research sponsors. Papers are reproduced by The Rand Corporation as a 
courtesy to members of its staff. 
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(1) The more minutes of instruction the pupil receives, the more he 
learns. 

(2) Instructional specialists, trained as they are in diagnostic 
individualized instruction techniques, are likely to be more effective 
in teaching reading, per minute of instruction, than other teaching 
personnel. 

(3) Pupils learn more in programs in which diagnostic, instruction- 
al, evaluation, and administrative personnel are well coordinated con- 
cerning objectives. 

(h) Since differences in the family life style of the pupils have 
possible important effects upon their motivation and ability to learn, 
socio-economic characteristics should be controlled as carefully as 
possible. 

The data gathering effort was focused upon information needed to 
test the hypotheses just enunciated. Questions were asked concerning 
the amounts and types of instructional resources used as well as the 
coordination or "teamwork" present. Information was a 1 so gathered for 
several variables meant to measure socio-economi c status , including 
mobility, racial characteristics, and percent of children in the school 
attendance area receiving aid to families with dependent children. ^ 

STATISTICAL TECHNIQUES USED 

The statistical procedure used was mul ti regression analysis, a 
technique through which it is possible to estimate the effect of one 
variable while at the same time holding constant the effects of other 
variables in the model. Since it was felt that instructional time, 
program coordination, and pupil socio-economic characteristics were 
important a priori as discussed above, it was important to include vari- 
ables for each of these effects in the explanatory model. It was also 

necessary to account for the effects of program length and pupil be- 
[i^l 

ginning score. Beyond these considerations, variables were chosen 
or omitted from the model on the basis of their explanatory power. 

Various combinations of functional forms were tried for the variables, 
but usually the linear form turned out to be best. 
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The model which was consistently best in explanatory power included 
the following variables: 

• Beginning Score 

• Program Length 

• Percentage of Pupils Who Belong to a Minority Group 

• Minutes of Instruction by Reading Specialists^ 

• Minutes of Instruction by Paraprofess ionals Helping Regular 
Classroom Teachers 

• Percentage of Instruction in a Separate Facility 

• Hours of Planning per Week 

This model was fit to pooled reading achievement data for all pupils 
in grades 2, 3> *», and 5 as wel 1 as for grade 3 alone. (Grade 3 was the 
only single grade for which there were enough observations to allow 
fitting the equation separately.) For the pooled data all variables in 
the model had explanatory power; ■ for grade 3> however, only beginning 
score, program length, and special ist instruction were relating to reading 
gains at high probability levels. 

SUMMARY OF FINDINGS 

Table 1 gives the relationships of the key resource-using variables 
to gain in the pooled grade scores along with some cost estimates. Out- 
comes for other variables are listed in Table 2. 

It is important to note the figures in Column k of Table 1, the 
probability that the results did not occur by chance. This gives what 
might be considered an estimate of the dependability of the findings, 
with the higher the figure, the higher the probability that we would 
discover a similar relationship in a different set of data. Most re- 
searchers do not feel confident about this assumption unless the prob- 
ability is 90 percent, and much preferably, 95 percent. Thus the findings 
in Table 1 for paraprofess ionals assisting the classroom teacher and use 
of a separate facility, while seeming to show that these inputs are 
efficient, are certainly not dependable enough for us to accept without 

[ 7 ] 

strong reservations. 

The instructional variable which is most consistently and dependably 
(in the sense just mentioned) related to reading gains was that for the 
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trained reading specialist. Indeed, as already mentioned, special ist 
instruction was the only school input related to reading performance in 
the third grade considered separately with the strength of the relation- 
ship even stronger in that grade than in the four pooled grades. While 
the variables for instruction by paraprofess iona 1 s were less statisti- 
cally significant, they showed signs of having some importance. This 
was especially true for instruction by paraprofess ional instruction in 
support of the regular classroom teacher The only instructional 
variable which was never related to reading gains at all was that for 
the regular classroom teacher. It should be noted that in the schools 
studied the classroom teacher usually had other responsibilities besides 
those to program children and typically could not devote a few additional 
minutes of instruction per week to them. 

Of the variables constructed to measure planning and coordination, 
only that for hours of planning itself was related to performance. The 
socio-economic variable most related to reading gain was that denoting 
percentage minority (black, Spanish surname, American Indian) and even 
this relationship was weak. There was also a weakly significant inverse 
relationship between percentage of pupils in the project school who were 
target children and reading gain. This is further discussed below. 

To summarize the foregoing resul ts, the first two hypotheses given 
above are strongly supported. Minutes of instruction, especially those 
by the trained reading specialists, were consistently related to reading 
gains. The hypothesis concerning coordination is supported to the extent 
that we can interpret planning hours as a coordination variable, although 
the finding is not replicated when data for grade 3 alone are used. 
Finally, the variables used for home socio-economic characteristics did 
not reflect much difference in performance, with percentage minority and 
percentage target children being the only variables showing signs of 
being different from zero. 

COST AND EFFECTIVENESS 

Perhaps the best way to translate the findings of the study into 
results that are readily meaningful to policymakers is to present them 
in cost terms. This is done in Column 3 of Table 1. Presenting such 
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cost estimates, even if carefully constructed, is not without its dangers, 
and care must be taken that the figures are not interpreted too literal ly.^® 
They are perhaps best interpreted to mean that we may entertain a reason- 
able hope that $100 per pupil spent on instruction by reading specialists, 
working alone or in some combination with paraprofessional assistants, 
can return in the neighborhood of an additional one-tenth of gain per 
month of instruction. If true, this would mean that an additional expendi- 
ture of $300 per pupil would bring these children to a learning rate near 
the national norm, if we use the figure of 0.7 months gain as the "normal" 
learning rate of pupils meant to be reached by Title I programs, as many 
do. - In situations where the present system is failing, as is often 
alleged to be the case in many high minority core-city schools, it might 

be efficient to substitute specialist instruction for relatively large 

r 1 2 i 

amounts of traditional self-contained classroom instruction. J 
CONCLUDING COMMENTS 

It is widely believed, mostly on the basis of the reports of large 
national surveys, that compensatory education has "failed." The findings 
of this study, which demonstrate modest average success and the possi- 
bility of very respectable gains in reading if diagnostic reading special- 
ists are used for instruction, stand in partial contradiction to this. 

Supposing these findings are accurate (and this can be checked only 

through replication), why are the large survey results so different? I 

think the answer to this lies in the fact that the methodology used in 

such surveys has been dangerously faulty, as pointed out in a recent Rand 

mi 

survey of educational evaluation. The most glaring defect is that 

they depend on results of matching program children with control groups 
which are probably superior. Randomly matched control groups in research 
designs for ongoing Title I and Headstart programs are almost non-existent 
and conscientious educators almost always choose children for the program 
who need it most. In addition, it seems reasonable to assume that "spill- 
over" gains from program to non-program chi ldren take place. There- 

fore, while it is reasonable perhaps to assume that "no difference" 
findings from such research tell us that there are no large gains, it 
cannot be said that there are no gains. 
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On the other hand, there is increasing evidence from other compen- 
satory education research which tends to support the findings here 
Guszak discusses research which he feels gives rise to a "reasonable 
hunch" that instruction by diagnostic reading teachers is effective for 
disadvantaged pupils.^*** Bisseli has shown convincingly in a careful 
analysis of the findings of many wei i -designed compensatory education 
research projects that better learning rates are associated with the 
degree of external organization and sequencing of the child's learning 
experiences, hierarchical organization of objectives, a directive teacher 
role, and the nature and amount of program supervision and personnel 
training. 1 These attributes are precisely those that are present 
with instruction by trained specialists, especially so when the program 
is planned such that the regular classroom teacher and paraprofessionai s 
are well coordinated to the specialists' activity. 

At the time of writing (March 1972), there is considerable national 
controversy over the issue of whether quality education can be given in 
core city schools without resorting to large bussing programs. The 
present study has been used by some in that controversy to imply that 
such schools can be greatly improved with additional resources. ■ 

It is, in fact, dangerous to extrapolate the findings in this study 
to core city schools with one-hundred percent disadvantaged populations 
directly. There were six such schools in the study sample and none of 
the six used more than minimal instruction by reading specialists.^®^ 
This means that we cannot extrapolate the important specialist finding 
to such schools directly. Of course, we still have the finding that 
specialist instruction is related to reading gains in general, and there 
is no reason to believe that it may not hold true for core city schools 
even though we do not have any direct evidence that it does. 

There is other evidence which supports, but only weakly, the 
notion that program children do better in schools where their percentage 
is less than half of the total school enrollment. In the sample there 
were thirteen projects where average gain in reading was greater than 
one month for each month of instruction and all thirteen were projects 
where less than fifty percent of total school enrollment were program 
children. When a variable for percentage program children in the 
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building was used in the regression equations, it was found to be 

[ 19 ] 

negatively related to gain although not at high levels of probability. 

It would be incorrect to infer from this weak relationship that large 
gains can be scored only in buildings where less than half of the children 
were in the program, but the evidence does nonetheless lean slightly in 
that direction. 

There are some logistical problems in adding large amounts of 
specialist instruction to one hundred percent disadvantaged core city 
schools also. In an inner city elementary school with 600 pupils, for 
example, if each pupil were to get thirty minutes of attention from a 
specialist weekly, about fourteen specialists would be required. This 
means that additional facilities equal to more than half the size of the 
present building will be required, by no means a minor undertaking. On 
the other hand, if only one-quarter or one-third of the students in a 
school building are program children, facilities can often be found 
(portable classrooms, etc.) without major disruption and expense being 
necessary. 

One last caveat perhaps should not be necessary. The findings here 
are for only one subject, Reading, and then only in four elementary 
grades. They are not properly extrapolated to other subjects or other 
grades. They also have little to tel 1 as concerning how long they will 
be maintained, which is undoubtedly the single most important question 
dema nding our at tent ion . 



\ 



Table 1 (Cont'd) 






f 






8 * 



m 



7 







- 10 - 



Table 2 

The Relationship of Non-Resource Using or Non-Statistical ly 
Important Variables in the California Title I Study, 
by Relationship to Reading Gain a 
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Variables Related to Gain in Reading Score 
Program Length in Months from Pre-Test to Post-Test 

Average Score on Stanford Reading Test, All Program Children, Beginning 
of Program 

Percentage of Program Children American Indian, Black, or Spanish Surname 
(Pooled Grades at the .85 Probability Level Only) 

Percentage of Children in the School Building Who Are Title 1 Target 
Children (The .80 Probability Level Only) 



Variables Not Related to Gain in Reading Score 

Minutes of Instruction per Week by the Regular Classroom Teacher (Beyond 
the Normal Program) 

b 

Hours per Week of Formal In-Service Training by Instructional Personnel 

Dummy Variable for Whether or Not All Project Personnel Were Ultimately 
Directed by a Single Manager 

Dummy Variable for High and Low Use of a Full-Time Psychologist for 
Diagnosis 

Percentage of Pupils in Program at Beginning of Year Still Present at End 
of Program Year 

Percentage of Pupils in School Attendance Area Who Received Aid to Families 
With Dependent Children 



^A fuller description of the variables as well as the means and standard 
deviations for most of them is contained in the original report, pp. 17-25. 

This variable did not include informal (but highly important) in- 
struction in the nature of on-the-job training given by specialists and 
classroom teachers to paraprofessional s. This omission, plus the fact that 
specialists have thorough training in individualized instruction while 
other personnel do not, could account for the non-significance of this 
variable. 
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FOOTNOTES 



1. H. J. Kies ling, Input and Output in California Compensatory Education 
Projects, The Rand Corporation, R-781-CC/RC, Santa Monica, California, 
1971. 

2. The sample was chosen on a stratified random basis subject to the 
limitation that only projects which employed the Stanford Reading 
Test were considered for use. These accounted for about one third of 
all California projects in the 1969-70 school year. I know of no 

a priori reason why this limitation may have made the sample unrepre- 
sentative of the state. Stratification criteria used in sample se- 
lection were the percentage of children in the school attendance area 
receiving aid to families with dependent children (AFDC) , percentage 
black, and percentage with Spanish surnames. 

3. The questionnaire used is published as an appendix to the original 
report . 

A. See original report, pp. 17"l8 and A3. 

5. The minutes of instruction variable was constructed to have a common 
denominator of the number of minutes of instruction each child re- 
ceived per week on an individual equivalent basis. The following is 
an example of how the variable was constructed. If a single special- 
ist saw groups of 10 pupils 30 minutes per day five days per week. 
Individual Equivalent Minutes (lEM's) would be 15 (30 divided by 10 
times 5). If the spec ial ist has one paraprofess ional assistant for 
these ten pupils, lEM's for each, abstracting from supervision time, 
doubles. Since it is assumed that the specialist and the parapro- 
fessional both lose ten percent of their time in the specialist's 
supervision of the paraprofessional , lEM's for each was computed to 
be 13.5 and not 15. The convention used to account for time taken 

in supervision was to deduct ten percent of the instructional time 
of supervising teacher and paraprofessional for each of the first 
two paraprofessional aides, and five percent for each aide after that. 

6. Appendix B in the original report contains a discussion of the 
strengths and weaknesses of using pooled grade data. 

7. The probability for paraprofessional instruction assistance of the 
classroom teacher is conservatively stated, however. See the dis- 
cussion in the next paragraph. 

8. This finding fits some a priori notions enunciated in a paper by 

F. J. Guszak: "The Diagnostic Reading Teacher for the Disadvantaged 

Child," in J. L. Frost and G. R. Hawkas (eds.). The Disadvantaged 
Child, Houghton-Miff 1 in , Boston, 1970, pp. 361-378. 

9. The statistical significance given in Table 1 for this variable is 
conservatively stated because the variable was significant at higher 



13 






« 



7 



- 12 - 



10 . 



11 . 



12 . 



probability levels in some alternative model specifications besides 
the one used. 

A slight error in the original report should be noted here. When 
t e l°g ar| thm of the specialist instruction is used, a problem exists 

used *'?“ 109 0 = - 00 - N ° n,atter «*»* convention is 

used to deal with this, the instruction variable for paraprofess ionals 

helping classroom teachers is statistically significant at the 97 

2?“?! H T Ver> “ he " the U " ear ° f fe specialist var I- 

able is used significance drops to 75 percent. In Table 1 the value 

J " a J an ? vera 9 e of these two findings. The convention used for 
shio I r m t f e 1 ? r, 9 |na ! report overstates the functional relation- 
linear f the ? pec,al,st variables (especially near zero) and if the 

what ls wIV 5 TJl I?** 0Verstates the paraprofessional variable some- 
when fLr! * th ' S paper the conse rvat i ve choice was always made 
when there was some doubt about which functional form of the model was 

In no statistica 1 exercise such as this study can cause and effect 

?n the°r Sh i PS if e ^' nferr ^ d Wlth certaint y- This can only be approached 
doss i hi ^?" tr °J led experiment. This means that there is always the 
5? . fl ’ y tha 5 5 he relationships found might in fact be explained by 
influence of forces not being studied. As an example in the present 

afso Y h a nn rha r S E UP ' S . projects hav|n 9 more specialist instruction 
also happen to be members of families where there is more sophisti- 

interac V on * The c«u.eof better reading gains is funde- 

strictioS'n er n CeS m fam, I y c ' rcumstances and not specialist in- 
struction at all: our conclusion is erroneous. 

5' a a ;f ful reconsideration of the data for possible alternative 

reL?nn t,0 " S f ? P th ®. r ®! at ,onsh ' P between specialist instruction and 
.. 1 9 gain, I could find none that seemed plausible. One possibility 
“ d ? “ Ith / ha central city schools had I Ittle special l« 

But whnl 0l it a " d i ]S ° parfo ™ ed Poorly (see also the concluding section). 
But while it is true that inner city schools did not use specialists 

heavily, only two out of seven such schools were very far below the 
mean (one was slightly above). Besides this, regressions for the 35 
2?": " ne r c \% schools still showed the specialist variable to be 

so?fo-ecZlf! C H-^ StatiSti ? an )f' A ran9e ° f Possibilities involving 
socio economic differences in the home were also considered. If these 

ab? e r to ate ?i t0 i- a r y 0t tT soci o" econ o m ' c characteristics which I was 
ab e to collect formally or notice informally, the possibility that 

this explanation is true also seems small. Y 



The Coleman Report found an average gain of 0.75 months 
who hod not dropped out. Wargo and associates assume a 
months gain per month of instruction. 



for children 
rate of 0.67 



If Title I children, 20 to a classroom, were to be given individu- 

of l ^H 5 7- trU ? t J 0n . by d | a 9 n °s tic specialists three hours a day instead 

classro^ TrJ nS ?^ Ctl u n ^7 there are 30 pupMs in a self-contained 
*oAA Sr00m * calculate the difference in cost per pupil per vear at 

Ld?,-' Ti: This inC,Udes the two addition! eta sirens 

additional d,a 9 „ostlc and Instructional materials, and t£e useTone 



ERIC 



14 






vm 



7 







C; 






1 






-13- 



paraprofess i onal assistant to the specialist. It also assumes, of 
course, that the regular classroom teacher is used profitably else- 
where. 

13. H. A. Averch. S. J. Carroll, T. S. Donaldson, H. J. Kiesling, J. 

P incus, How Effective Is Schooling? The Rand Corporation, R-956- 
PCSF/RC, 1972, Chapter 6. 

14. See the discussion of the findings by Gray and Klaus in ibid. 

15. Op. cit. 

16. Joan S. Bissell , The Cognitive Effects of Pre-School Programs for 
Disadvantaged Children, Unpublished Doctoral Dissertation, Harvard 
University Libraries, 1970. 

17. See Testimony of Secretary Elliot Richardson before the Sub-Com- 
mittee on Education, Committee on Labor and Public Welfare, U.S. 
Senate, March 24, 1972, pp. 32-33* 

18. One had 21 minutes of instruction per child per week and the other 
five all had less than ten. Also see the note ten above. 

19. The probability that the coefficients obtained were different from 
zero was about .80. 
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